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In considering the apparent failure of the t%#o major 
compensatory education programs. Project Head Start and Title i, 
there are so many tinctxitrolled variables interacting simultaneously 
tJat it is virtually impossible to define the specific etiological 
factors engendering the results obtained. The Federal programs are 
difficult to evaluate because their goals are broad, they involve 
inillions of children, and they are administered by Federal agencies 
far away from indi.xdual projects, other problems are created through 
snch variables as program effects or maturation (our lack of 
knowledge about preschool learning and disadvantaged learners) , 
interactions of various socialising agencies, and technology. 
Reliability of measurement devices is especially doubtfuJ at the 
preschool level. Our lack of knowledge in the affective domain is 
even greater. The major weaknesses of the compensatory evaluation 
are: (1) lack of comparable groups and control groups; (2) no planned 
variation in programs; (3) lack of random selection and/or assignment 
of Ss to treatment and control groups; (4) lack of clear-cut criteria 
for inclusion in the program; (5) lack of clearly specified 
objectives; and (6) non-comparable data. Futiire intervention programs 
should adhere to the tenets of experimental research. Residential 
centers are recommended to remove disadvantaged children from 
impoverished envircmments in infancy. A bibliography is provided. 
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Introduction : Arthur Jensen (1969, p. 1) stated that, ''Compen- 
satory education has been tried and apparently it has failed." This 
statement is accurate if restricted to the overall results of the two 
major federally funded compensatory education programs (Project Head 
Start and Title I); however, it is a rather gross oversimplification 
of the problem* in a sense it is analagous to a research problem in 
that the results are not (or should not be) an end point, but, instead, 
a beginning or impetus to further investigation regarding '*why" the 
effort "failed" or "succeeded." (See Edwards, 1968) Jensen (1969) 
contended that the failure was due to reasons extrinsic to the programs 
themselves, whereas the present author claims that there are so many 
variables interacting simultaneously that it is virtually impossible to 
delineate the specific etiological factors engendering the results obt- 
ained. The author further contends that the "apparent failure" could 
be attributed co two other eoually as plausible factors, namely: (1) 
the actual implefaentation and design of the specific programs, or non- 
evaluational . actors, e.g. , type of program, and (2) the experimental 
design or evaluational considerations, e. g. , non-comparable groups. 
It should be noted that neither of these factors mmrn- independent or 
mutually exclusive of the other. 



Because It is readily acknowledged that the results of the two 
major compensatory efforts hava not been successful in their attempts 
to reach the program objectives (See Mc Dill, et al., 1969; Cohen, 1970; 
and Westinghouse-Ohio Study, 1969), the author will not analyze the 
specific findings in detail. The impcrtitnt point is not that these 
programs have '*f ailed" but, rather, the many problems they have con- 
fronted (ranging froro the political to the non-comparability of groups) 
and which have thus seived at, competing alternative hypotheses. Further 
discussion of the general and specific problems confronting these pro- 
grams will illustrate the author's contention regarding the difficulty 
in delineating the many oossible etioloitf ca'' factors contributing to 
the results obtained. 

General Factors : Cohen (1970) indicated that prior to 1964 
educational evaluation had been primarily confined to small scale re" 
search in which the purpose of the study was generally limited to rather 
specific factors and typically involved a small budget and staff. How- 
ev^*", after 196^ the federal government became involved in establishing 
broad educational programs which Cohen (1970, ?13) perceived as differ- 
ing from the previously conducted research in three important ways; 

(1) they are social action programs, and as such arc not 
focused narrowly on teachers' in-service training or 
on a science curriculum, but aim broadly at improving 
education for the disadvantaged. 

(2) the pew programs are directed not at a school or a 
school district, but at millions of children, in 
thousands of schools in hundreds of pchool juris- 
dictlonc in all the states; 



(2) they arc not ccncerned and executed by a teacher, 

principal, a superintendent, or a researcher - they 
were created by the Congress and are administered 
by federal agencies far from the school districts 
which actually design and conduct the individual 
projects. 

Without delineating all the ruestions and implications engendered by 
the abovf it is obvious that any large scale program will create many 
problems. For example, how does one ef fee lively evaluate the specific 
effects upon approximately three million children spread out across 
the nation? Is it reasonable to evaluate on the basis of critera 
related primarily to achievement for programs directed at rather 
broad, sweeping political, economic, and social changes? Should evalua- 
tion be decentralized despite the fact that they are national programs? 
How does one determine the specific effects of any undertaking when the 
overall objectives for the program are determined nationally, but yet 
each local school district (or state agency) is responsible ior the 
actual implementation of the program? These are but a few of the 
important Questions that could be raised and as Cohen (1970, p. 215) 
stated, "In the social action programs, however, the political import- 
ance of infoiTBation is raised to a high level by the broader political 
character of the programs themselves." The important point bcine that 
while the basic tenets of experimental research may be similar for ev- 
aluating both small and large scale programs, i. e. , determining their 
effects, the important difference lies in the character of the aims and 
organization of the programs. Campbell (1969, p. 410) reached conclusions 
similar to Conens^Stating that, "If the political and administrative 
system hai coaiaitted itself in advance to the correctness and efficacy 



of ita reforms. It cannot tolerate learning of failure. To be truly 

scientific ve must be able to experiment. We must be able to advocate 

without that excess of commitment that blinds us to reality testing.'* 

For example, one would logically assume that some type of evaluatlonal 

procedure would be involved in order to assess whether or not a program 

has been effective, but as Cohen (1970, p. 219) states: 

The mandate for evaluation - like many Congressional 
authorizations • lacked any cnablinf'. »Tiechanism: re- 
sponsibility for carrying out the evaluation was 
specifically delegated to the state and local education 
authoritites who operated the programs. It was not 
hard to see, in 1965, tliat this vas eouivalent to 
abandoning much hope of useful program evaluation. 

Campbell (1969) indlcate»r^ that oiany feel we are at the point of 

continuing or discontinuing programs on the basis of assessed effective- 

ness. He Questions the validity of this attitude when he states that^ 

"most ameliorative programs end up with no interpretable evaluation." 

Another example is the fact that Title I programs are funded on a formula 

grant type basis^ in which the amount of money given to any educational 

district is based on how many poor children. ±n the district has ^^nrolled 

in the schools, and not on how well the district may or may not educate. 

The actual implementation and evaluation of these programs are thus 

confounded by many non-evaluational considerations; for example, 

politically vested interests on various levels and the emotionally 

laden overtones of such programs. For a more detailed and complete 

discussion of other factors one is directed to Campell, 1969; Timpan ^ 

1970; Cohen, 1970; Mc Dill, et al., 1969. 



■'0 



Another problem confronting compensatory education programs, 
specifically at the preschool level, has to do with the type of vari- 
ables with which an investigator trust cope, Mc Dill, et al. (1969, 
p, 7) cites three important variables or factors which affect co^np^n• 
satory programs: namely, program effects or maturation, interactions 
of various socializing agencies, and technology. 

Many programs^directed at preschr-ol and elementary school children^t^ 
based on a common belief among some educators that the earlier we begin 
assisting children of this age the more successful we may be. (For 
example, ree Bloom, 1964) The problem this creates is that we have 
accumulated much more knowledge of the learning process and the effects 
of other variables upon children in the elementary school, relatively 
•peaking, than those that affect preschool children. Only in recent 
years have efforts been made to study this much younger population. 
According to Mc Dill, et al. (1969, p. 7), "Compensatory education or 
no compensatory education, we simply do not know much about how pre- 
school children learn, and we know even less about disadvantaged learners. 
Because of this it is difficult to determine whether the programs them- 
selves are ineffective, or whether they are ineffective because of our 
inability to define the critical variables in order to assess the impact 
of the program, Campbell and SLanley (1966) discuss a related problem 
when they list maturation as a potential confounding variables which 
might possibly effect the internal validity of an experiment. They 



ask the oueation, ''How does one distinguish between maturation and 
treatment effects in young children?'' Research strategies relating to 



the intervention caotiel proposed by many will naturally fae affected by 



a lack of scientific knowledge concerning intellectual and general cog* 
nitive processes. Compensatory education as a strategy is not in oues- 
tion but rather the theorectical structure which supports the decisions 
that implement such a program. The present state of knowledge and the 
problems it creates for those interested In assessing the impact of 
various programs remains an obstacle to certainty in aesec^sjitnt . Gener- 
ally speaking, researchers attempt to select one point in time as the 
input and another as the output, but research does not indicate if the 
two points are necessarily the most important in the life cycle of the 
individual. It may be that the significant factors have occurred prior 
to the experimental treatment (a problem, by the wt^y, in all research), 
and is one reason why many recommend that program implementation be 
begun in infancy. (For example, see Bloom, 1964; Hechinger, 1966; and 
Gladkowskiy 1971) The important point is that we do not actually know 
whether our programs have the effect they are designed to have. As 
Zimllies (1969, p. 179) stated: 

The problem, then, is reduced to finding the appropriate 
inputs for achieving the desired output. While schema- 
tically this may appear to be an accurate analysis of the 
problem, it by passes the critical intervening and moisting 
factor - the child. No where does one find a deacription 
of the four-year-old child, a developmental analysis of the 
personality and cognitive functioning children a.^ this 
age level, or a statement of their primary areas of conflict, 
typical modes of resolution, and principle spheres of develop- 
ment. 

Interaction between socializing agencies represents another source 
of difficulty for evaluation. This problem evolves around the fact that 
education does not take place exclusively in the schools. A child may 
be Involved in a formal educational program for six hours per day with 
the other eighteen hours representing no controlled educational activity 
from schools. Does the remaindei' of the time outside the program cancel 



any potentially positive effects that titight have occurred during thr 
treatment? Is there an optimum amount of time spent tn school which 
could be effective? The answers to theee fuestions are, of course, 
not available at ^he present time, although th#w are questions which 
will eventually need answers if we are to identify and aasess the 
effectiveness of our programs, 

Gordon (1970) presented an excellent overview of various attempts 
to assist disadvantaged segments of our society in which he provides a 
brief synopsis concerning the areas of concern and directions for appro- 
aching the problems in program implementation for the di >5adv8ntaged , Much 
of the difficulty of explanation and interpretation of the various 
positions arise due to a confounding of factors in an attempt at delinea- 
tion. For example, it has been phown that as Southern Blacks move North 
their achievement levels increase. The question arises, however, as to 
whether this is due to tVie impact of the school, selective migration, 
non-school environmental conditions, the interaction of these factors, 
or others not yet investigated. The interaction of marv factors increases 
the complexity of attemptv*? at explaining any outcome of an intervention 
effort. (For example, see ^'rotberg, 1969) 

According to Mc Dill, et al. (1969), if one had a firm idea of the 
relevant variables important to any program design one would still be 
jFaced with the fjuestion of measurement. How much can we rely on our 
measurement devices to give as the data we need for evaluating outcomes' 
The difficulty arises at all levels but even more so at the preschool 
level because of the relatt* e lack of measurement data concerning this 
age group. The younger the child the m%*re inaccurate our measurment 



devices are likely to be. For example, if a child were tested at age 
two on one of the standardized infant scales available we would not 
expect as high a correlation with later achievement as we would i f we 
were to adminirter the tfst at later a^es. Mc Dill, et al. (\9o9) 
Indicates that whil^ the state of developn^cnt regarding coj«?nit1ve 
dimensions is still "primitive" the picture is even more depressing 
when one considers the affective domain. (See Wick and R^ggs, 19/1; 
Cronhach, 1960; and Me^^rens and ^.ehmann, 1970 for a more complete 
discussion of the many factors involved in the evaluational process) 

The discussion presented above concerned itself with general 
factors affecting evaluations] research. This section will delineate 
some of the more specific research problems (f».g., lack of control 
groups) relating to compensatory education programs. 

One of the primary difficulties inherent in compensatory programs 
has been an obvious lack of CQj^itrol over relevant variables, ranging 
from no n- comparable groups for comparisons, (no control groups in certain 
instances), to the Interaction effects of the environment, (Mc Dill, et 
al., 1969) For example, the evaluation of Project Head Start ccntiiined 
many factors which were uncontrolled in the design. First, randomlv 
selected experimental or control groups were not used but instead an 
€X-po8t-facto design in vhich the controls were selected and matched 
after the experiment als had i^lready received th^ treatment constituted 
the basis for. the evaluation, \'his, of course, makes it impossible to 
determine the specific effects of the program and thus violates one of 
the basic tents of experimental research. It should he indicated that 
the evaluators of Project Heil Start did randomly select the centers 



for the study but this was Invalidated by the previously cited weak- 
nesses inherent in the assessment of various local programs. The 
following factors were cited as representative of these weaknesses- 
<^We6tlnp,house-0hlo Study, 1969) 

1. Lack of cooiparablHty amon;! separate and Indeoeiident 
studies becau'-e of dlffer'*nt enrollment criteria, 
program treatments, design, instrumentation, and 
schedules for gathering data. 

?. Tn some cases the absence of aiy comparison proup. 

Too few cases, freouently only those enrolled at 
a particular center. 

4. Geographical restrictions to local or regional groups. 

On the basis of these difficulties selecting a "random" sample of an 

already biased or non-comparable sample 'kes not eliminate the sources 

of bias. 

Second, there were no uniform or standr'rdi zed procedures adiiered 
to between various pro?;rams to Insure that tne f»valuatlon would be 
att- -opting to assess those factors which programs shared in common. 
For example, the various centers emploved somewhat different goals, 
treatments, and program procedures, thus masking between and within 
center differences. Some centers were In opeVitlon for two hours per 
day whereas others were In opf ration for tour hours; some centers were 
only in Operation for two months whereas others were in operation for 
eight or nine months out of the year. ^See Cohen, 1970 and hfc Dill, et 
aU, 1969) Despite th^se differences the programs were all evaluated 
•">s i^f they were similar; however » there is no way of ascertaining which 
specific centers were relatively successful as compared to those which 
were not. Regarding this masking effect Cohen (1970, p. 2?6) stated 
that, **The problem, then, is not only to identify what thr programs 



delUver, but also to •ystematicAlly exprriment with strategies for 
affecting achool outcomes* ♦ ♦The movement toward experltner.tatlon presumes 
that the most efficient way to proceed Is systematic trial and discard, 
discovering and repeating effective strategies/* Others vho hold 
similar views regarding ''planned variations" include Smith and Mght 
(1970) and Campbell (1969). This approach was not employed In the 
Head Start Project although the evaluative team did recoranend this 
for future consideration. 

In the assessment of Project Head Start the emphasis was on '"over- 
all" effectiveness of the program, disregarding those centers which might 
have been particularly effective. What this would mean in practice is 
that if a center (or certain aspects of a center) were found to be par- 
ticularly effective then one could further investigate it in order to 
determine how it differs from the other centers or programs in its oper- 
ation. If significant differences were detected then other centers could 
be organised in which the be«t features of proven programs could be 
incorporated, as well as the fact that presently operating programs could 
^.hus be modified. 

Other weaknesses which contributed to the overall evaluational efforts 
Included lack of ui ^formity across the various centers regarding such 
matters as the use of the same indices of measurement, objectives of the 
program, and the selection criteria of Sm for treatment and control groups. 
This uniformity had not been accomplished in many of the programs because, 
in part, the local programs were permitted the freedom to not only evaluate 
their own program but also to decide upon a specific implementation 
course. As stated by Cohen (1970, p. 227), "The Office of Education... 
does not reouire that the same tests be used in all Title I projects. 



indeed, it doe« not require that an^r teiti uft^." In order for 
an appropriate evaluation to be undertaken iuch matters 3i this mist 
be CO sidered before the Implementation of the program, thus obviating 
later problems arising regarding interpretation of the resoltc. 

Many of the weaknesses inherent in the experimental designs are 
those related to internal validity, that is, those factors associated 
with the cuestion: Did the experimental treatments make a difference 
in this specific experimental instance? fSei; Campbell and Stanley, 1966) 
With so many weaknesses in evidence it is virtually impossible to answer 
this question. Hence the studies undertaken to date are of very limited 
scientific value in determining whether or not the programs ere effective. 
The following comprises the major weaknesses of compensatory evaluations 
and would thus servt as a rather formidable list of competing alternative 
hypotheses to any research undertaking: 

1. Lack of comparable groups, and in some cases, no 
control groups at all. 

2. Nc planned variation in programs in order to assess 
both within and between center differences. 

3. Lack of random selection and/or assignment of Ss 
to treatment and control eroups, 

4. Lack of clear cut criteria for inclusion into the 
program. 

5. Lack of clearly specified objectives. 

6. Kon-comparable data, i.e., different indices of 
measurement. 



S ijrnna r yj_ The myriad of probl'^ms have not proven amenable to 
solutions with the delineation of the specific factors contributing 
to the problem being an almost impossible task due to the tnanv un- 
controlled variables interacting, siraultaneouslv , thereby con^^oudning 
both process and expected results, '^he attempts at assessm^rit have 
consistently violated the sine oua nons of experimental lesearch, with 
the evidence regarding the effectivness of our programs being ambiguous, 
(For example, see Cohen, 1970; Mc Dill, et ai,, 1969; Campbell, 1969: 
Posner, 1968; Etzioni, 1968; Hyman and Wright, 1967), In response to 
the initial oucstion posed the author thus agrees with Jensen (1969) 
that the programs leave a great deal to be desired, although does 
not agree with his premise. The fact is that the specific etiological 
factors remain aonewhat problematic. 

Alternative Strategy : "iven this st^e of aff^if^s the author 
is recommending an alternative strategy designed to improve upon two 
very important diaienslons of any compensatory program, namely, the 
evaluational and programmatic. 

Firct, an approach advocated by Campbell (1969) and Campbell and 
Erlebacher (1970) Id which they recommend that future intervention 
programs adhere to the basic underlying tenets of experimental research 
and vhich closely approximate "true" experimental designs, e. g. , random 
selection and assignment of Ss to treatment and control groups. (See 
Campbell and Stanley, 1966 for a more complete discussion of what is 
meant by "true" experimental designs and the underlying tenets of 
experimental research) 



Second, implementing residential centers which would employ £ 



total human development approach of assistance for children and 
farailles from impoverished ir.ilieus by removing the child in i'>.fancy 
(or a verv early age) therebv: ^* 

1, Providing an opportunity to assist a child in an 
environment in which he is relatively free of the 
pressures and tensions which exist (or might) in 
his usual social and family life, and which make 
difficult ( or impossible) an evaluation of all 
Che factors that contribute to his piobl^ms. 



2. Providing an opportunity for the parents of the 
child to receive the needed assistance ,e .g • , 
vocational and/or educational training, while 
care is being provided for the children. 



The author believes that the strategy recommended would improve 
considerably upon the evaluational and programmatic dimensions of an 
intervention program when compared with our previous efforts. The 
implications of employing a residential approach evolve around the 
concept of "control." That is, by placing children in such centers 
It would be possible to exert considerablv more control over the 
various environmental contingencies and would thus minimize the here- 
tofore uncontrolled sources of variance. Second, it would be a program 

which would assist both child ^"'^ n^rent simultaneously thereby meetin 

the needs of all parties concerned. 

1. For s complete discussion of this strategy see the author's thesis 
entitled "Residential Human Development Centers to Assist Children 
and Families from Deprived Milieus," submitted to the faculty of 
Northwestern University, June, 1971. 



B5 blio^ raphy 



**loco), Stabi llty and fhange in r '^an Characteristics , N<^u ^o-rk- 

John Wilpy and Sons, Tnc, , 19^A, 

CampV^^ll, r. T, "Reform as Fxperirapnts , " An.i«rican Psy cho 1 is , 
April, 1^69, Volume 2^, Number 4. 

Campbell, 7,, and Frlebacher, A., ''Paw Repressioa Arrifacts in 

Ouasi-Fyperimental Evaluations Can Mistakenly Make Compensatory 
Kducation Look Harmful,** in Hellmuth, J, fed.)* Compensatory 
Education : A National Debate, Volume 1 1 1 of the Disadvantaged 
Child . New York: Brunner-Mazel , 1970. 

Campbell, D. T., and Stanley, J, C. Experimental and OMasi-Experimental 
Designs for Research . Second Printing. Chicago: Rand McNally and 
Company, 1966. 

Cohen, D. "Politics and Rese-irch: Evaluation of Social Action 

Programs in Education," Review of Educational Research , April, 1970, 
Volume 40, Number 2, 213-258. 

Cronbach, L. J. Essentials of Psycholof^ical Testing . New York: Harper 
and Row, 1960. 

Edwards, A. L. Experimental Design in Psychological Research. Third 
Edition. New York: Holt, Rinehart and Winston, 1968. 

Etzioni, A. "Shortcuts to Social Change'^", The p-^>^Mc Interest , 1968, 
12, 40-51, 

Gladkowski, G. J. "Residential Human Development Centers to Assist 
Children and Families from Deprived Milieus." (Doctoral thesis 
submitted to the faculty at Northwestern University, Evanston, 
Illinois, June, 1971) 

Gordon, E* W. "Introduction," Review of Educational Research , February, 
1970, Volume 40, Number 1. 

Grotberg, E. Critical Issues in Research Related to Disadvantaged 
Children . Princeton, New Jersey: Educational Testing Service, 
1969. 

Hcchinger, F. M. Preschool Education Today . New York: Doubleday and 
Company, 1966 » 

Hyman, H^ H., and Wright, C. R. "Evaluating Social Action Programs," 
in Lazarsfeld, P. H*, et al» (edsO* The Uses of Sociology . New 
York: Basic Books, 1967. 



Jensen, A. '"^'ow Much Can W Boos^ IQ and Scholastic Achievement?/' 

Harvard F ducatlonal Reviev , Reprint Series No. ?. Gcttr-^er, i^G^. 

Mr Pill, F, L., V oill, ^. S,, and ?prehe, J. T. Strarev^ies for 
Sucess in Compensator y Kducation • An Appraisal of Fva luai i O' 
Research , Baltimore, Maryland: The ,'ohns I^opkins Press, 19t^^. 

Mehrens, W. A,, and I.ehtnann, I. J. Standardized Tests in Fducacion . 
New York: Hole, Rinehart and Winston, Inc., 1969. 

Posner, J. Kvaluations of "Successful" Pro iects in Compensatory 

Education. U. S. Of f ice. of Fducatlon, Office of Planning and 
Evaluation. An occasional paper. Number v^, April, 1968^ 

Smith, P. v., and Light, R. J. "Choosing a P'uture: Strategies for 
Designing and Evaluating New Programs,*' Harvard Educational 
Review, February, 1970, Voltjme 40. Nitmher 1. 

Timpane, P. M. "Educational Experimentation la National Social 

Policy," Harvard Educational Review, Novemeber, 1970, Volume 40, 
Niflnber 4, 

Westinghouse Learning Corporation-Ohio University. "The Impact of 

rieadstart: An evaluation of the Effects of Head Start on Children's 
Cognitive and Affective Development," EFIjC Document Resume . VD 036 
321, June, 1969. (Executive Summary) 

Wick, J. , And Beggs, D. L. Evaluation for Decision Making in the 
Schools. Boston; Houghton Mifflin Company, 1971. 



Zimilies, H. ••Review of Martin Deutsch^s Book, The Disadvantaged 
Child," Harvard Educational Review , 1969, 39, 177-180. 



